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ABSTRACT 


Each  of  the  domains  of  HSI  is,  of  itself,  a  discipline  with  vast  amounts  of 
research,  analytic  techniques,  educational  programs,  and  methods  for  evaluating  the 
effectiveness  of  the  system  with  respect  to  the  specific  domain.  Relatively  recently, 
domains  with  a  logical  similarity  have  been  the  focus  of  interest  for  researchers  studying 
the  plausibility  of  creating  evaluative  tools  which  take  into  account  the  constraints  of 
multiple  domains.  This  interest  has  led  to  the  creation  of  various  tools  with  which 
acquisition  professionals  can  more  accurately  determine  the  impact  of  design  decisions 
on  the  system  as  a  whole.  However,  no  single  tool  has  yet  been  created  which  takes 
into  consideration  the  constraints  of  all  the  domains  which  HSI  encompasses.  The 
development  of  such  a  tool  would  give  decision-makers  the  ability  to  quickly  and 
accurately  determine  the  system-wide  trade-offs  associated  with  changes  in  a  single 
domain. 

In  order  for  this  to  occur,  an  in-depth  study  of  the  current  tools  associated  with 
each  of  the  HSI  domains  must  be  conducted.  The  most  accurate  tools  from  each 
domain  must  be  integrated  with  a  single  interface.  However,  this  step  will  only  be 
realized  after  a  common  language  has  been  identified  which  can  speak  to  the 
effectiveness  of  the  system  in  each  of  the  domains.  Finally,  the  human  interface  with  the 
tool  must  be  intuitive,  and  designed  with  the  end-user  in  mind. 

This  study  identified  the  various  resources  currently  available  for  evaluating  each 
of  the  HSI  domains.  These  resources  were  compiled  in  a  searchable  database  for  use 
by  the  HSI  professional  in  the  planning  of  HSI  evaluations.  Following  a  description  of 
how  HSI  relates  to  the  Department  of  Defense  acquisition  process,  the  design  effort  to 
produce  an  overarching  interface  was  presented.  This  interface  would  allow  the 
acquisition  professional  to  evaluate  the  trade-offs  between  all  relevant  domains  and 
make  well-informed  decisions  with  respect  to  the  overall  effectiveness  of  the  human  in 
the  system.  Next,  a  plan  for  insertion  of  the  process  and  software  into  the  acquisition 
community,  making  the  tool  available  to  all  acquisition  professionals,  was  discussed. 
Finally,  as  with  all  research,  the  limitations  of  the  present  study  were  discussed,  as  well 
as  recommendations  for  future  research. 
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I.  INTRODUCTION 


A.  HUMAN  SYSTEMS  INTEGRATION 

In  its  broadest  sense,  Human  Systems  Integration  (HSI)  is  the  effort  to 
make  human  considerations  the  top  priority  in  systems  design.  This  broad 
definition,  however,  does  not  escape  the  grasp  of  the  more  global  study  of 
systems  engineering.  A  system  is  defined  as  “a  group  of  interacting,  interrelated, 
or  interdependent  elements  or  parts  that  function  together  as  a  whole  to 
accomplish  a  goal”  (Massachusetts  DOE,  2001).  The  basis  of  systems 
engineering — systems  thinking — requires  that  any  product  or  service  be  viewed 
as  a  piece  of  a  larger  system  in  order  to  better  understand  its  requirements  and 
characteristics.  Sage  and  Armstrong  (2000)  describe  systems  engineering  as 
having  three  key  components:  organizational,  technology-based,  and  humans. 
As  some  areas  of  responsibility — and  the  interactions  between  the  three 
components — tend  to  involve  more  than  a  single  component,  visualizing  systems 
engineering  as  a  Venn  diagram  may  prove  useful  (see  Fig  1). 


SYSTEMS  ENGINEERING 


Figure  1 .  Three  key  areas  of  Systems  Engineering 

Each  of  the  key  components  of  systems  engineering  falls  under  the 
purview  of  existing  disciplines.  While  systems  engineering  efforts  incorporate  all 
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three  components,  the  products  differ  from  that  of  other  engineering  disciplines. 
Systems  engineering  produces  abstract  systems,  and  relies  on  the  other 
engineering  disciplines  to  design,  build  and  test  the  tangible  products.  These 
well-defined  disciplines  have  explicit  knowledge  of  their  specific  domains  and 
thereby  design  products  to  exacting  details. 

The  organizational  component  of  systems  engineering  may  be  viewed 
through  the  discipline  of  macroergonomics.  Macroergonomics  grounds  itself  in 
sociotechnical  systems  theory,  with  the  chief  concern  being  human-organization 
interface  technology — specifically,  the  analysis,  design,  and  evaluation  of  the 
work  system.  Dr.  Hal  W.  Hendrick  (2002) — considered  the  “father”  of 
macroergonomics — describes  the  discipline’s  goal  as  “[optimizing]  the  work 
system’s  design  in  terms  of  its  sociotechnical  system  characteristics”  and 
ensuring  that  these  characteristics  are  carried  down  throughout  the  design  of  the 
sub-system  efforts. 

The  traditional  engineering  disciplines  are  encompassed  by  the 
technology-based  component  of  systems  engineering.  Practitioners  in 
disciplines  such  as  electrical,  mechanical,  and  aerospace  engineering  apply  their 
expertise  in  designing  systems  to  meet  detailed  specifications.  While  these 
specifications  ensure  the  proper  functioning  of  the  system  with  respect  to  each 
individual  discipline,  they  do  not  necessarily  ensure  the  system  will  function 
properly  with  the  additional  systems  with  which  it  may  need  to  operate.  This 
level  of  integration  is  left  to  systems  engineers. 

The  final  component  in  systems  engineering — humans — falls  under  the 
purview  of  HSI.  Where  systems  engineering  focuses  on  the  defining,  developing 
and  deploying  of  large  scale  systems,  HSI  champions  the  human  element  within 
systems  design.  It  has  been  argued  that  focusing  on  the  human  element  of  a 
system  is  the  most  likely  method  for  increasing  system  performance  and 
reducing  system  life-cycle  costs  (Booher,  2002).  The  exact  boundaries  and 
areas  of  responsibility  of  HSI  are  not  yet  completely  defined.  However,  the 
defense  acquisition  structure  rests  comfortably  upon  the  foundation  of  HSI. 
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B.  BRIEF  HISTORY  OF  HUMAN  SYSTEMS  INTEGRATION 

Just  as  the  trade  space  for  systems  engineering  incorporates  multiple 
fields  of  study,  the  present  trade  space  for  Human  Systems  Integration 
coordinates  the  efforts  of  several  distinct  disciplines  or  domains.  These  domains 
from  which  HSI  has  emerged  have  existed  for  many  years.  Though  known  by 
different  names,  and  having  evolved  over  time,  the  fields  incorporated  in  HSI 
have  always  tried  to  answer  the  same  question — how  can  the  human  be  more 
effective  in  the  system  ultimately  resulting  in  enhanced  overall  system 
performance.  In  fact,  the  predecessors  of  HSI  in  the  U.S.  military  date  back  to 
the  early  years  of  the  Army  Air  Corps,  and  the  efforts  to  improve  the  performance 
of  systems  (aircraft)  specifically  at  the  points  where  humans  interfaced  with  the 
machine  (Chapanis,  1959).  This  initial  effort  resulted  in  the  birth  of  human 
factors.  More  detailed  histories  of  human  factors  can  be  found  by  Meister  (1999) 
and  Boff  (2006). 

Within  the  past  seventy  years,  the  field  of  human  safety  has  also  gained 
prominence,  as  the  modern  consumer-driven  society  required  more  standardized 
practices  and  better  products  (Stephens  &  Rowan,  2006).  In  1946,  Amos  L. 
Wood  presented  the  first  formal  presentation  on  what  is  now  considered  system 
safety  (Miller,  1966). 

Both  of  these  well  established  areas  of  study  began  as  grass-roots  efforts 
to  improve  the  effectiveness  of  the  human  interacting  with  the  rest  of  the  system. 
From  the  early  days  of  human  factors  and  system  safety,  additional  efforts  have 
arisen  for  better  integrating  human  and  machine,  especially  with  respect  to 
military  systems  acquisition.  More  recent  efforts  include  the  MANPRINT  effort  in 
the  U.S.  Army  (U.S.  Army,  1990),  and  what  is  now  being  termed  Human  Systems 
Integration — a  Department  of  Defense  (DoD)  level  multi-service  effort  to 
champion  the  human  in  systems  with  which  they  interact.  Among  other 
achievements,  this  latter  effort  has  resulted  in  the  establishment  of  a  Master  of 
Science  in  HSI  available  through  the  U.S.  Naval  Postgraduate  School  (NPS)  at 
Monterey,  CA. 
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C.  HSI  DOMAINS 

While  an  in-depth  history  of  HSI  can  be  found  in  Booher’s  (2002) 
Handbook  on  Human  Systems  Integration,  the  present  study  requires  only  an 
understanding  of  the  fields — or  domains — encompassed  by  HSI  and  the  inherent 
interactions  between  these  domains.  The  MANPRINT  program  identified  seven 
key  domains  within  its  purview:  manpower,  personnel,  training,  human  factors, 
system  safety,  health  hazards,  and  soldier  survivability  (U.S.  Army,  2000).  The 
MANPRINT  handbook  makes  note  of  the  existence  of  the  interactions  between 
domains;  however,  the  extent  of  the  information  provided  consists  of  the 
following  statement  and  some  brief  examples: 

Although  each  of  the  MANPRINT  domains  has  been  introduced 
separately,  in  practice  they  are  often  interrelated  and  tend  to  impact 
on  one  another.  Changes  in  system  design  to  correct  a  deficiency 
in  one  MANPRINT  domain  nearly  always  impact  another  domain 
(MANPRINT  Directorate,  2005). 

Building  upon  the  MANPRINT  effort,  the  Human  Systems  Integration 
program  identified  an  additional  area  of  specific  importance  to  the  U.S.  Navy,  but 
with  reasonable  application  to  all  services — habitability  (U.S.  Naval  Postgraduate 
School,  2006).  Additionally,  the  NPS  Master’s  program  focuses  on  the 
interactions  between  domains,  recognizing  these  interactions  and  trade  offs  as  a 
distinct  requirement  for  the  HSI  professional. 

As  the  discipline  of  HSI  is  continually  refined,  it  is  possible  that  the  number 
and  definitions  of  HSI  domains  may  increase,  decrease,  or  evolve.  However, 
since  the  aforementioned  eight  domains  presently  constitute  the  study  of  HSI,  a 
brief  description  of  each  is  given  below.  The  definitions  provided  are  taken 
directly  from  the  MANPRINT  and  NPS  HSI  programs,  and  from  the  DAU  Defense 
Acquisition  Guidebook. 

1.  Manpower 

Manpower  addresses  the  number  of  military  and  civilian  personnel 
required  and  potentially  available  to  operate,  maintain,  sustain,  and 
provide  training  for  systems  in  accordance  with  Section  2434  of 
Title  10,  U.  S.  Code.  It  is  the  number  of  personnel  spaces  (required 
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or  authorized  positions)  and  available  people  (operating  strength).  It 
considers  these  requirements  for  peacetime,  conflict,  and  low 
intensity  operations  (MANPRINT  Directorate,  2005). 

Manpower  [comprises]  the  actual  number  of  men  and  women,  in 
the  military  as  well  as  civilian  and  contractor  personnel,  required  to 
operate  and  maintain  military  systems,  including  those  personnel 
who  support  and  provide  training  for  the  users  of  military  systems. 
Considers  the  impacts  of  automation  on  both  manpower  utilization 
rates  and  on  military  operator-to-seat  ratios  in  both  system 
operation  and  maintenance  (NPS,  2006). 

Manpower  factors  are  those  job  tasks,  operation/maintenance 
rates,  associated  workload,  and  operational  conditions  (e.g.,  risk  of 
hostile  fire)  that  are  used  to  determine  the  number  and  mix  of 
military  and  DoD  civilian  manpower  and  contract  support  necessary 
to  operate,  maintain,  support,  and  provide  training  for  the  system. 
Manpower  officials  contribute  to  the  Defense  acquisition  process  by 
ensuring  that  the  program  manager  pursues  engineering  designs 
that  optimize  manpower  and  keep  human  resource  costs  at 
affordable  levels  (i.e.,  consistent  with  strategic  manpower  plans). 
Technology  approaches  and  solutions  used  to  reduce  manpower 
requirements  and  control  Lifecycle  costs  should  be  identified  in  the 
capabilities  documents  early  in  the  process.  For  example,  material¬ 
handling  equipment  can  be  used  to  reduce  labor-intensive  material¬ 
handling  operations  and  embedded  training  can  be  used  to  reduce 
the  number  of  instructors  (USD/(AT&L),  2006). 

2.  Personnel 

Personnel  addresses  the  cognitive  and  physical  characteristics  and 
capabilities  required  to  be  able  to  train  for,  operate,  maintain,  and 
sustain  materiel  and  information  systems  (MANPRINT  Directorate, 
2005). 

[The]  personnel  [domain  consists  of]  the  cognitive  and  physical 
capabilities  necessary  for  the  training,  operation,  maintenance,  and 
support  of  military  systems.  Includes  the  attitudes,  experiences, 
and  other  human  characteristics  necessary  to  achieve  optimal 
system  performance  by  matching  the  “right  person”  with  the  “right 
job”  (NPS,  2006). 

Personnel  factors  are  those  human  aptitudes  (i.e.,  cognitive, 
physical,  and  sensory  capabilities),  knowledge,  skills,  abilities,  and 
experience  levels  that  are  needed  to  properly  perform  job  tasks. 
Personnel  factors  are  used  to  develop  the  military  occupational 
specialties  (or  equivalent  DoD  Component  personnel  system 
classifications)  and  civilian  job  series  of  system  operators, 
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maintainers,  trainers,  and  support  personnel.  Personnel  officials 
contribute  to  the  Defense  acquisition  process  by  ensuring  that  the 
program  manager  pursues  engineering  designs  that  minimize 
personnel  requirements,  and  keep  the  human  aptitudes  necessary 
for  operation  and  maintenance  of  the  equipment  at  levels 
consistent  with  what  will  be  available  in  the  user  population  at  the 
time  the  system  is  fielded  (USD/(AT&L),  2006). 

3.  Training 

Training  is  defined  as  the  instruction,  education,  on-the-job,  or  self 
development  training  required  providing  all  personnel  and  units  with 
essential  job  skills,  and  knowledge  (MANPRINT  Directorate,  2005). 

Training  [includes]  the  instruction,  education,  and  “on-the-job 
training”  necessary  to  provide  personnel  with  the  requisite 
knowledge,  skills,  and  abilities  needed  for  the  correct  and  safe 
operation  and  maintenance  of  military  systems  across  a  wide  range 
of  operational  conditions  (NPS,  2006). 

Training  is  the  learning  process  by  which  personnel  individually  or 
collectively  acquire  or  enhance  predetermined  job-relevant 
knowledge,  skills,  and  abilities  by  developing  their  cognitive, 
physical,  sensory,  and  team  dynamic  abilities.  The 
"training/instructional  system"  integrates  training  concepts  and 
strategies  and  elements  of  logistic  support  to  satisfy  personnel 
performance  levels  required  to  operate,  maintain,  and  support  the 
systems.  It  includes  the  "tools"  used  to  provide  learning 
experiences  such  as  computer-based  interactive  courseware, 
simulators,  and  actual  equipment  (including  embedded  training 
capabilities  on  actual  equipment),  job  performance  aids,  and 
Interactive  Electronic  Technical  Manuals  (USD/(AT&L),  2006). 

4.  Human  Factors 

The  goal  of  HFE  [Human  Factors  Engineering]  is  to  maximize  the 
ability  of  an  individual  or  crew  to  operate  and  maintain  a  system  at 
required  levels  by  eliminating  design-induced  difficulty  and  error. 
Human  Factors  engineers  work  with  systems  engineers  to  design 
and  evaluate  human-system  interfaces  to  ensure  they  are 
compatible  with  the  capabilities  and  limitations  of  the  potential  user 
population  (MANPRINT  Directorate,  2005). 

Human  Factors  Engineering  [is]  the  comprehensive  integration  of 
human  characteristics  (both  physical  and  psychological)  into  the 
definition,  design,  development,  and  evaluation  of  military  systems 
in  order  to  optimize  performance  in  human-machine  interactions. 
This  includes  human  interaction  with  products,  equipment,  systems, 
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and  environments,  and  addresses  the  capabilities  and  limitations  or 
personnel  during  this  interaction  (NPS,  2006). 

Human  factors  are  the  end-user  cognitive,  physical,  sensory,  and 
team  dynamic  abilities  required  to  perform  system  operational, 
maintenance,  and  support  job  tasks.  Human  factors  engineers 
contribute  to  the  Defense  acquisition  process  by  ensuring  that  the 
program  manager  provides  for  the  effective  utilization  of  personnel 
by  designing  systems  that  capitalize  on  and  do  not  exceed  the 
abilities  (cognitive,  physical,  sensory,  and  team  dynamic)  of  the 
user  population.  The  human  factors  engineering  community 
integrates  the  human  characteristics  of  the  user  population  into  the 
system  definition,  design,  development,  and  evaluation  processes 
to  optimize  human-machine  performance  for  both  operation  and 
maintenance  of  the  system  (USD/(AT&L),  2006). 

Human  factors  engineering  is  primarily  concerned  with  designing 
human-machine  interfaces  consistent  with  the  physical,  cognitive, 
and  sensory  abilities  of  the  user  population  (USD/(AT&L),  2006). 

5.  System  Safety 

System  Safety  is  the  design  features  and  operating  characteristics 
of  a  system  that  serve  to  minimize  the  potential  for  human  or 
machine  errors/failures  that  cause  injurious  accidents  (MANPRINT 
Directorate,  2005). 

System  Safety  [is]  the  design  of  machine  and  system  features  that 
minimize  the  potential  for  human  or  machine  errors,  and  of  human 
and  machine  failures  that  can  cause  injuries.  Also  included  is  the 
ability  of  the  system  to  be  operated  and  maintained  without  injury  to 
personnel  or  to  other  equipment  (NPS,  2006). 

Safety  factors  consist  of  those  system  design  characteristics  that 
serve  to  minimize  the  potential  for  mishaps  causing  death  or  injury 
to  operators  and  maintainers  or  threaten  the  survival  and/or 
operation  of  the  system.  Prevalent  issues  include  factors  that 
threaten  the  safe  operation  and/or  survival  of  the  platform;  walking 
and  working  surfaces  including  work  at  heights;  pressure  extremes; 
and  control  of  hazardous  energy  releases  such  as  mechanical, 
electrical,  fluids  under  pressure,  ionizing  or  non-ionizing  radiation 
(often  referred  to  as  "lock-out/tag-out"),  fire,  and  explosions 
(USD/(AT&L),  2006). 

6.  Human  Survivability 

Soldier  survivability  addresses  the  characteristics  of  a  system  that 
can  reduce  fratricide,  detectability,  and  probability  of  being 
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attacked,  as  well  as  minimize  system  damage,  soldier  injury,  and 
cognitive  and  physical  fatigue  (MANPRINT  Directorate,  2005). 

Human  Survivability  [is]  the  ability  of  personnel  to  exist  and  function 
during  and  following  exposure  to  hostile  environments  or  situations. 
[Survivability]  includes  issues  involving  enemy  and  friendly  combat 


weapons-induced  injuries  and  the  inherent  hazards  to  personnel 
during  threat/combat  conditions,  and  the  inherent  hazards  of 
military  equipment  (NPS,  2006). 

Personnel  survivability  factors  consist  of  those  system  design 
features  that  reduce  the  risk  of  fratricide,  detection,  and  the 
probability  of  being  attacked;  and  that  enable  the  crew  to  withstand 
man-made  hostile  environments  without  aborting  the  mission  or 
suffering  acute  chronic  illness,  disability,  or  death  (USD/(AT&L), 
2006).  ” 

7.  Health  Hazards 

Health  Hazards  addresses  the  design  features  and  operating 
characteristics  of  a  system  that  create  significant  risks  of  bodily 
injury  or  death.  Along  with  safety  hazards,  an  assessment  of  health 
hazards  is  necessary  to  determine  risk  reduction  or  mitigation 
(MANPRINT  Directorate,  2005). 

Health  Hazards  [includes]  the  identification  of  risk  factors  in  military 
systems  and  the  physical  environment  that  can  increase 
opportunities  for  system-caused  bodily  injury  or  death.  This 
includes  many  inherent  conditions  present  in  the  operation,  use, 
and  maintenance  of  a  system  (e.g.,  heat,  cold,  shock,  recoil, 
motion,  vibration,  toxic  fumes,  chemical  &  biological  agents,  noise, 
radiation,  etc.)  that  can  reduce  job  performance  and  contribute  to 
injury,  illness,  or  death  (NPS,  2006). 

Occupational  health  factors  are  those  system  design  features  that 
serve  to  minimize  the  risk  of  injury,  acute  or  chronic  illness,  or 
disability;  and/or  reduce  job  performance  of  personnel  who  operate, 
maintain,  or  support  the  system.  Prevalent  issues  include  noise, 
chemical  safety,  atmospheric  hazards  (including  those  associated 
with  confined  space  entry  and  oxygen  deficiency),  vibration, 
ionizing  and  non-ionizing  radiation,  and  human  factors  issues  that 
can  create  chronic  disease  and  discomfort  such  as  repetitive 
motion  diseases.  Many  occupational  health  problems,  particularly 
noise  and  chemical  management,  overlap  with  environmental 
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impacts.  Human  factors  stresses  that  create  risk  of  chronic  disease 
and  discomfort  overlap  with  occupational  health  considerations 
(USD/(AT&L),  2006). 

8.  Habitability 

Habitability  [addresses]  the  physical  living  environment  in  which 
personnel  are  required  to  live,  work,  and  sleep  while  performing 
their  military  duties  during  peace  and  war.  This  includes  the 
physical  and  psychological  needs  of  the  individual  and  group,  and 
takes  into  account  morale  and  the  social  environment  during  both 
sustained  and  continuous  military  operations  (NPS,  2006). 

Habitability  consists  of  those  characteristics  of  systems,  facilities 
(temporary  and  permanent),  and  services  necessary  to  satisfy 
personnel  needs.  Habitability  factors  are  those  living  and  working 
conditions  that  result  in  levels  of  personnel  morale,  safety,  health, 
and  comfort  adequate  to  sustain  maximum  personnel  effectiveness, 
support  mission  performance,  and  avoid  personnel  retention 
problems  (USD/(AT&L),  2006). 

Habitability  factors  are  those  living  and  working  conditions  that  are 
necessary  to  sustain  the  morale,  safety,  health,  and  comfort  of  the 
user  population.  They  directly  contribute  to  personnel  effectiveness 
and  mission  accomplishment,  and  often  preclude  recruitment  and 
retention  problems.  Examples  include:  lighting,  space,  ventilation, 
and  sanitation;  noise  and  temperature  control  (i.e.,  heating  and  air 
conditioning);  religious,  medical,  and  food  services  availability;  and 
berthing,  bathing,  and  personal  hygiene  (USD/(AT&L),  2006). 


D.  DESCRIPTIVE  ANALOGY  OF  HSI 

An  analogy  may  help  illustrate  the  interactions  among  the  HSI  domains 
and  the  benefit  of  a  tool  which  would  allow  acquisition  professionals  to  visualize 
the  effects  of  tradeoffs  between  domains.  Take,  for  example,  the  light  bulb. 
There  are  a  number  of  necessary  elements  which  constitute  an  ordinary 
incandescent  bulb:  the  glass  bulb,  the  contents  of  the  glass  bulb,  the  filament, 
and  an  electric  current.  While  Thomas  Edison  and  many  additional  scientists 
and  inventors  of  the  1800’s  knew  the  individual  properties  of  each  of  these 
elements,  it  took  over  100  years  to  advance  from  Humphrey  Davy’s  1809 
charcoal  strip  arc  lamp  to  the  incandescent  bulb  in  use  today.  The  significant 

advancements  occurred  only  when  the  interactions  between  the  elements  were 
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taken  into  consideration.  While  Davy’s  charcoal  strips  produced  intense  light, 
they  burned  out  quickly.  The  enclosure  of  a  filament  in  an  evacuated  tube 
increased  its  longevity.  Altering  the  composition  of  the  filament  produced  a  wide 
range  of  bulb  life.  Add  to  this  the  injection  of  different  gasses  into  the  previously 
vacuous  bulb,  and  the  filament  lasted  even  longer  (Arizona  State  University, 
2006). 

As  with  most  design  considerations,  there  were  tradeoffs  to  consider  in  the 
development  of  the  light  bulb.  Such  tradeoffs  are  best  understood  by  a  more 
detailed  description  of  the  latest  advancements  in  incandescent  bulbs.  Filaments 
burn  out  as  a  result  of  evaporation,  or  sublimation,  during  use.  Also,  in  order  to 
produce  the  proper  level  of  light,  filaments  have  to  be  heated  to  a  certain 
temperature.  In  order  to  increase  filament  life  by  reducing  evaporation,  the  bulb 
was  filled  with  inert  gasses  such  as  nitrogen  and  argon.  However,  this  addition 
altered  the  heating  characteristics  of  the  filament.  The  inert  gasses  dissipated 
heat  so  efficiently  that  the  filament  could  not  produce  a  satisfactory  level  of  light. 
An  additional  change  had  to  be  made  in  order  to  restore  the  effectiveness  and 
efficiency  of  the  light  bulb.  Researchers  accomplished  this  by  making  a  small 
alteration  to  the  physical  characteristic  of  the  filament — winding  it  into  a  coil.  This 
procedure  allowed  enough  heat  to  build  up  around  the  coil  to  produce  the 
necessary  light  level  before  the  inert  gasses  could  cool  the  filament. 

In  a  similar  fashion,  acquisition  professionals  observe  the  necessary 
elements  of  an  acquisition  program — one  of  which  is  inevitably  the  human — and 
seek  the  most  efficient  and  effective  combination  of  these  elements  which  will 
meet  the  stated — and  often  unstated — requirements. 

It  is  interesting  to  note  that  modern  incandescent  light  bulbs  are  not 
energy  efficient.  Today’s  available  technology  is  advancing  the  concept  of  using 
highly  efficient  light  emitting  diodes  (LEDs)  to  produce  equivalent  levels  of  light 
with  a  fraction  of  the  energy  involved  in  traditional  incandescent  bulbs.  While  the 
concept  is  essentially  the  same — passing  electric  current  through  a  substance  to 
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produce  light — modern  technology  is  using  the  available  resources  in  the  most 
effective  and  efficient  manner  to  date. 

E.  SCOPE  OF  PRESENT  STUDY 

Each  of  the  domains  of  HSI  is,  of  itself,  a  discipline  with  vast  amounts  of 
research,  analytic  techniques,  educational  programs,  and  methods  for  evaluating 
the  effectiveness  of  the  system  with  respect  to  the  specific  domain.  Relatively 
recently,  domains  with  a  logical  similarity  have  been  the  focus  of  interest  for 
researchers  studying  the  plausibility  of  creating  evaluative  tools  which  take  into 
account  the  constraints  of  multiple  domains.  This  interest  has  led  to  the  creation 
of  various  tools  with  which  acquisition  professionals  can  more  accurately 
determine  the  impact  of  design  decisions  on  the  system  as  a  whole.  However, 
no  single  tool  has  yet  been  created  which  takes  into  consideration  the  constraints 
and  interactions  of  all  the  domains  which  HSI  encompasses.  Crisp,  Hoang, 
Karangelen  and  Britton  (2000)  emphasize  the  importance  of  technologies  which 
support  such  a  total  system  design,  especially  with  respect  to  the  human 
operator.  With  respect  to  HSI  evaluation,  this  concept  may  prove  to  be  difficult,  if 
not  impossible,  to  realize.  The  inherent  differences  of  each  system  may  call  for  a 
different  set  of  tools  to  produce  an  accurate  evaluation.  In  fact,  some  systems 
may  require  the  development  of  system-unique  tools  to  properly  evaluate  the 
human-system  effectiveness.  However,  an  overarching  interface  which  allows 
for  the  inclusion  of  any  number  and  variety  of  resources  utilized  in  the  evaluation 
process  would  be  an  invaluable  tool  to  the  HSI  professional.  The  development  of 
such  a  tool  would  give  decision-makers  the  ability  to  quickly  and  accurately 
determine  the  system-wide  trade-offs  associated  with  changes  in  a  single  area. 
In  order  for  this  to  occur,  a  comprehensive  study  of  the  current  tools  associated 
with  each  of  the  HSI  domains  must  be  conducted.  Following  this  study,  it  will  be 
necessary  to  identify  how  outputs  from  the  tools,  which  come  in  a  variety  of 
metrics,  can  be  normalized  in  a  useful  manner.  This  step  will  only  be  realized 
after  a  common  interface  has  been  defined  which  can  be  configured  to  effectively 
convey  the  dynamic  relationships  between  decision  parameters.  Finally,  the 
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overarching  interface  must  be  properly  disseminated  to  those  conducting  HSI 
evaluations.  Of  course,  as  is  the  goal  of  all  efforts  involving  HSI,  the  interface 
must  be  intuitive,  and  designed  with  the  end-user  in  mind. 

The  purpose  of  the  present  study  was  to  describe  a  process  for 
conducting  HSI  evaluations  and  design  the  necessary  resources  and  software  to 
conduct  such  evaluations.  The  study  identified  the  various  resources  currently 
available  for  evaluating  each  of  the  HSI  domains.  These  resources  were 
compiled  in  a  searchable  database  for  use  by  the  HSI  professional  in  the 
planning  of  HSI  evaluations.  Following  a  description  of  how  HSI  relates  to  the 
Department  of  Defense  acquisition  process,  the  design  effort  to  produce  an 
overarching  interface  was  presented.  This  interface  would  allow  the  acquisition 
professional  to  evaluate  the  trade-offs  between  all  relevant  domains  and  make 
well-informed  decisions  with  respect  to  the  overall  effectiveness  of  the  human  in 
the  system.  Next,  a  plan  for  insertion  of  the  process  and  software  into  the 
acquisition  community,  making  the  tool  available  to  all  acquisition  professionals, 
was  discussed.  Finally,  as  with  all  research,  the  limitations  of  the  present  study 
were  discussed,  as  well  as  recommendations  for  future  research. 

This  chapter  has  introduced  the  concept  of  HSI,  defining  the  common 
domains  associated  with  HSI,  and  discussed  the  purpose  of  the  present  study. 
The  following  chapter  discusses  the  effort  conducted  to  collect  information  on 
HSI  evaluation  resources,  and  the  result  of  this  effort — a  database  for  HSI 
analysts  to  use  in  planning  HSI  evaluations. 
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II.  HSI  RESOURCES 


A.  DATA  COMPILATION 

Compiling  the  information  on  resources  available  to  HSI  professionals  is 
somewhat  complicated.  There  are  many  characteristics  by  which  the  resources 
can  be  identified.  While  these  characteristics  may  include  such  items  as  cost, 
manufacturer,  and  other  less  pertinent  details,  the  characteristics  most  relevant 
to  a  better  understanding  of  their  usefulness  to  the  HSI  professional  were 
selected  as  data  points.  The  standardized  format  for  data  collection  is  provided 
as  Figure  2.  For  the  most  part,  the  data  collection  headings  are  self  explanatory 
and  require  no  further  description.  Three  headings,  however,  benefit  from 
additional  discussion.  These  headings  are  Cross-Domain  Utility,  Readiness,  and 
Classification. 


Figure  2.  HSI  resource  data  collection  format 
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1.  Classification  of  HSI  Resources 

There  are  myriad  ways  to  classify  the  resources  available  to  HSI  analysts. 
The  classification  method  selected  needed  to  provide  the  user  with  the  most 
relevant  information  based  on  the  purpose  for  which  the  resources  were 
classified.  In  the  review  of  resources  conducted  for  this  study,  a  number  of 
common  terms  were  initially  identified.  These  terms  captured  a  majority  of  the 
available  resources,  and  are  listed  below  with  a  brief  description  on  how  each  is 
differentiated  from  the  others. 

Tool:  something  (as  an  instrument  or  apparatus)  used  in 

performing  an  operation  or  necessary  in  the  practice  of  a  vocation 
or  profession. 

Method:  a  systematic  procedure,  technique,  or  mode  of  inquiry 
employed  by  or  proper  to  a  particular  discipline  or  art. 

Model:  a  description  or  analogy  used  to  help  visualize  something 
(as  an  atom)  that  cannot  be  directly  observed. 

Database:  a  usually  large  collection  of  data  organized  especially 
for  rapid  search  and  retrieval  (as  by  a  computer). 

Questionnaire:  a  set  of  questions  for  obtaining  statistically  useful  or 
personal  information  from  individuals.  (Merriam-Webster,  2006) 

The  definitions  above  describe  types  of  resources  in  a  general  sense.  As 
one  of  the  products  of  the  present  study  would  be  a  useful,  searchable  database 
of  resources  for  HSI  professionals,  a  more  detailed  classification  of  these 
resources  would  allow  the  HSI  professional  to  conduct  a  more  refined  search  for 
the  appropriate  resource.  Lockett  and  Powers  (2003)  describe  a  classification 
process  for  tools  pertaining  specifically  to  the  field  of  Human  Factors 
Engineering.  Tools  were  classified  as  follows: 

•  guidelines  and  standards 

•  checklists 

•  subjective  assessment  tools 

•  simulations — unmanned  (with  multiple  subcategories) 

•  simulations — human  in  the  loop 

•  miscellaneous  analytical  tools 
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This  classification  process  was  slightly  modified  and  implemented  in  the 
compilation  of  the  Human  Factors  Engineering  Technical  Advisory  Group  (HFE 
TAG)  Directory  of  Design  Support  Methods  (DDSM),  which  is  reviewed  in  greater 
detail  below  (personal  communication  with  Teresa  Alley,  2006).  The  final 
classification  structure  used  in  the  DDSM  was  as  follows: 

•  Guidelines  and  Standards 

•  Checklists 

•  Subjective  Assessment  Tools/Surveys 

•  Perceptual  Models 

•  Simulation  -  Unmanned 

•  Task  Network/Workload  Tools 

•  Cognitive  Process  models 

•  Graphic  Human  Models 

•  Human  Behavioral  Representations  (HBR)  in  Simulation 
Federations 

•  HFE  Tools  Embedded  in  CAD/CAE  Suites 

•  Simulation  -  Human-in-the-Loop 

•  Integrated  Tools 

•  Government  Courses  and  Handbooks 

•  Design  Shells 

•  Information  Service  Center 

•  Databases 

•  Other 

Since  a  majority  of  the  resources  reviewed  in  this  study  were  present  in 
the  DDSM,  it  was  determined  that  the  same  classification  structure  would  be 
utilized  in  the  present  effort  as  well. 

2.  Cross-Domain  Utility 

One  of  the  key  tenets  of  HSI  is  an  understanding  of  the  interactions  that 

exist  between  the  identified  domains.  Individual  resources  were  categorized  by 

how  they  assist  in  the  evaluation  of  the  specific  domain  for  which  they  have  been 

designed.  However,  there  exists  the  potential  that  a  resource  may  provide 

additional  information  relevant  to  other  domains  for  which  the  resource  was  not 
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initially  designed.  Also,  the  domains  of  HSI  are  not  firmly  set  and  resources  may 
very  well  have  been  in  existence  longer  than  the  present  field  of  HSI.  It  therefore 
becomes  necessary  to  view  these  resources  not  only  with  respect  to  their  domain 
specific  relevance,  but  more  importantly,  in  how  each  resource  assists  the  HSI 
analyst  in  evaluating  the  overall  effectiveness  of  the  system.  The  resulting 
information  strengthens  the  HSI  analyst’s  ability  to  most  accurately  define  the 
relationships  between  the  relevant  domains. 

3.  Readiness 

A  necessary  consideration  in  selecting  a  resource  to  use  is  how  quickly 
the  resource  can  be  ready  for  use.  For  example,  a  commercially  available 
resource  may  be  purchased,  installed  and  ‘ready-for-use’  in  a  matter  of  hours. 
However,  utilizing  the  resource  in  an  effective  manner  may  require  the  user  to 
attend  an  extensive  training  program  or  to  purchase  additional  required 
resources.  With  a  clearer  understanding  of  the  ancillary  requirements  associated 
with  a  particular  resource,  the  selection  process  would  be  more  effective.  Left 
out,  the  decision  could  result  in  costly  overruns  of  both  budget  and  schedule,  as 
well  as  a  reduction  of  the  overall  effectiveness  of  the  HSI  analysis. 

B.  CURRENT  REPOSITORIES 

It  has  already  been  noted  that  each  domain  of  HSI  is  a  fully  developed 
discipline.  As  such,  lists  of  resources  for  specific  domains  are  prevalent.  A 
majority  of  these  refer  to  the  Human  Factors,  Safety,  Manpower,  Personnel  and 
Training  domains,  while  the  relatively  newer  areas  of  Habitability,  Survivability, 
and  Health  Hazards  provide  few  lists.  Locating  and  including  every  list  of  HSI 
domain  resources  may  be  virtually  impossible.  However,  many  of  the  most 
prominent  lists  from  around  the  world  have  been  included  in  this  study.  The 
following  are  descriptions  of  each  of  these  well-established  repositories. 

1.  DDSM 

The  Directory  of  Design  Support  Methods  (DDSM)  began  as  an  effort  of 
the  Designing  for  the  User  Subgroup  of  the  Department  of  Defense’s  Human 
Factors  Engineering  Technical  Advisory  Group  (HFE  TAG).  The  DDSM  is  a 
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living  document,  updated  continuously  to  provide  the  most  accurate  information 
on  design  support  tools  and  techniques  developed  by  agencies  such  as  DoD, 
NASA  and  the  FAA,  as  well  as  those  developed  by  NATO  countries,  academia 
and  the  private  sector  (DTIC-ASD,  2006).  The  online  document  is  updated  by 
the  office  of  the  Defense  Technical  Information  Center-A  in  San  Diego,  CA 
(DTIC-ASD).  A  site  map  of  the  DTIC  DDSM  website  is  included  as  Figure  3.  In 
addition  to  the  current  resources  listed  in  the  DDSM,  those  resources  which  are 
no  longer  supported,  have  transitioned  into  newer  tools,  or  are  proprietary  and 
accessible  only  through  contractual  support — while  not  included  in  the  report — 
are  available  through  an  online  archive  (Teresa  Alley,  personal  communication, 
Sep  29,  2006). 


Figure  3.  DDSM  site  map 

The  DDSM  is  a  great  resource  for  the  experienced  HSI  user,  and  it 
provides  a  wealth  of  readily  accessible  and  current  information  on  methods  used 
in  HSI  efforts.  The  DDSM  uses  the  aforementioned  classification  structure 
described  by  Lockett  and  Powers  (2003).  The  standard  information  collected  on 

each  method  of  the  DDSM  includes  the  method  title,  contact  information,  a 
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general  overview,  required  equipment,  input/output/processing  information, 
documentation  including  references  in  academic  and  professional  journal 
articles,  alternative  approaches,  the  method’s  stage  of  development,  date  of 
current  version  release,  and  validation  studies.  With  such  accurate  and  detailed 
information,  the  HSI  user  can  more  efficiently  select  the  appropriate  methods 
(DTIC-ASD,  2006). 

There  are,  however,  limitations  to  the  DDSM.  The  list  is  not  in  a  database 
format  and  there  are  no  search  options.  While  originally  compiled  using  a 
database,  the  database  files  were  lost,  leaving  only  the  text  format  used 
currently.  The  user  must  scan  the  entire  alphabetized  list  of  methods  or,  in  the 
case  of  a  novice,  search  each  record  for  the  appropriate  method.  Fortunately, 
the  current  version  of  the  DDSM  is  scheduled  to  be  updated  before  the  end  of 
FY2007,  with  the  intention  of  rebuilding  the  database  (Teresa  Alley,  personal 
communication,  Sep  29,  2006). 

2.  FAA  Human  Factors  Workbench 

The  Federal  Aviation  Administration  operates  a  Human  Factors 
Workbench  from  their  official  website  which  boasts  over  three  hundred  human 
factors  research  tools  and  techniques.  The  resources  are  grouped  into  the 
following  categories:  Physical  Ergonomics,  Human  Factors  Knowledge, 

Knowledge  Elicitation,  Human-Computer  Interaction,  Data  Analysis,  Modeling 
and  Simulation,  Human-System  Performance,  Program  Planning,  and  Safety. 
Each  of  these  categories  is  further  broken  down  in  a  structure  unique  to  each 
category.  The  FAA  Human  Factors  Workbench  tools  and  techniques 
categorization  is  provided  as  Figure  4. 
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Figure  4.  FAA  Fluman  Factors  Workbench  site  map 


3.  EuroControl 

As  part  of  the  Programme  for  Performance  Enhancement  in  European  Air 
Traffic  Management  (EATM),  EUROCONTROL  manages  a  database  of  tools 
and  methods  of  human  factors  which  are  used  to  resolve  tasks  related  to  HSI. 
This  list  is  available  online  through  the  EATM  Quick  access  link  on  Human 
factors  Integration  in  Future  Air  traffic  management  systems  (HIFA).  HIFAdata, 
and  EUROCONTROL  in  general,  specifically  target  HSI  activities  related  to  air 
traffic  management  systems.  However,  as  indicated  on  their  website,  the  tools 
and  methods  can  be  applied  to  the  development  of  “any  other  human-machine 
system”.  The  structure  of  the  HIFAdata  website  is  broken  out  similar  to  that  of 
the  FAA.  General  headings  of  Lifecycle,  Tasks,  Checks,  Methods,  Tools, 
Domains,  Roles,  References,  and  Glossary  provide  the  HSI  user  with  a 
convenient  classification  structure.  Additionally,  information  on  specific 
resources  in  the  HIFAdata  database  is  linked  to  all  relevant  categories.  This 
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database  design  allows  the  user  to  identify  the  most  appropriate  resource 
through  multiple  avenues.  A  breakdown  of  the  HIFAdata  classification  structure 
is  provided  in  Figure  5. 


Figure  5.  EUROCONTROL  HIFAdata  site  map 


4.  HSIIAC 

The  Human  Systems  Integration  Information  Analysis  Center  (HSIIAC) 
supports  research  and  development  efforts  in  HSI.  This  resource  began  in  1988 
as  an  effort  by  the  Defense  Logistics  Agency,  and  has  evolved  over  time  to  its 
current  configuration.  As  of  2005,  the  HSIIAC  is  managed  by  the  Air  Force 
Research  Laboratory’s  Human  Effectiveness  Directorate,  operated  under 
contract  to  Northrop  Grumman.  HSIlAC’s  mission  is  to  provide  analysis  services 
in  support  of  research,  design  and  development  of  defense  crew  systems. 
Among  the  information  on  the  HSIIAC  is  a  list  of  available  products  and  services. 
The  majority  of  these  are  informative  books  or  reports  detailing  methods  for 
conducting  human  systems  analysis.  However,  there  are  some  models  and  tools 

20 


listed  directly,  such  as  the  NASA  TLX  and  a  variety  of  anthropometric  data  sets. 
The  website  is  designed  to  provide  HSI  resources  by  taxonomy,  though  at  the 
present  time  few  resources  are  included.  The  overall  structure  of  the  HSIIAC 
website  is  provided  as  Figure  6. 


Figure  6.  HSIIAC  HSI  products  and  services  site  map 

5.  Defence  Research  and  Development  Canada 

The  Canadian  Department  of  National  Defence  (DND)  began  serious 
research  efforts  in  HSI  in  1998.  DND  initiated  an  effort  to  identify,  define,  and 
coordinate  the  HSI  process  as  part  of  the  HSI-Process  Models  project  under  the 
Defence  Research  and  Development  Canada  (DRD  Canada).  In  answer  to  the 
original  objective  to  “document,  demonstrate,  and  continually  enhance  a  set  of 
HSI  analysis  tools  and  techniques  as  well  as  models,  simulations  and  related 
databases”  (Greenley,  2000),  DRD  Canada  developed  an  online  repository  of 
HSI  information. 
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Focusing  mainly  on  the  HSI  process,  the  project  limits  its  list  of  tools  to 
describing  those  used  specifically  by  the  DND  for  HSI  analysis,  while  providing 
links  to  additional  lists  and  resources  outside  the  DND.  Unfortunately,  the 
descriptions  are  somewhat  limited,  and  only  a  DRD  Canada  contact  is  provided  if 
further  information  is  desired.  However,  beyond  the  limited  tools  listed,  the  site 
also  contains  abundant  information  on  the  HSI  process  as  conducted  by 
Canadian  Forces.  As  much  of  this  information  is  valuable  to  the  development  of 
HSI,  it  is  included  as  part  of  the  website  description  provided  in  Figure  7.  The 
site  is  in  the  process  of  being  moved,  and  will  soon  be  the  responsibility  of  the 
acquisition  framework  of  the  DND  (Shaw,  personal  communication,  Nov  9,  2006). 


6.  ONR  Science  and  Technology  Manning  Affordability  Initiative 

The  Office  of  Naval  Research  began  a  partnership  with  the  acquisition 
community  in  1998  to  conduct  research  with  the  end  goal  of  optimizing  systems 
for  the  warfighter  while  at  the  same  time  reducing  system  costs  and  improving 
system  performance.  In  a  similar  fashion  to  what  is  being  done  in  the  present 
study,  one  of  the  products  of  the  Manning  Affordability  Initiative  was  a  process 
which  would  “be  used  as  a  roadmap  for  identifying  and  (where  required) 
developing  tools  and  capabilities  for  the  S&T  project’s  Human-Centered  Design 
Environment”  (DD  21 /ONR,  1998).  The  process  resulted  in  a  collection  of  tools 
linked  to  the  six-step  systems  engineering  process  as  it  relates  to  human 
engineering.  The  steps  include  Mission  Analysis,  Requirements  Analysis, 
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Function  Analysis,  Function  Allocation,  Design,  and  Verification.  A  webpage  is 
devoted  to  each  tool  and  provides  a  general  description,  the  related  step,  and 
contact  information  for  the  tool  proprietor.  Additionally,  many  of  the  tools  include 
a  link  to  a  more  detailed  description.  The  detailed  description  answers  a  host  of 
questions  pertaining  to  the  use  and  functionality  of  the  tool: 

•  What  analyses  does  it  conduct?  What  does  the  tool  do? 

•  What  questions  does  the  tool  answer? 

•  How  complex  is  the  tool? 

•  What  fidelity  does  the  tool  have  to  have? 

•  What  experience  does  the  user  have  to  have? 

•  What  are  the  tool  inputs,  and  who  provides  them? 

•  What  are  the  tool  outputs  and  who/what  uses  them? 

•  What  platform  does  the  tool  currently  run  on?  What  language  is  it 
written  in?  How  is  the  data  stored? 

•  What  other  support,  infrastructure,  or  tools  are  required? 

While  the  website  is  a  great  resource  for  HSI-related  tools,  it  also  contains 
excellent  information  on  the  human  engineering  process  as  it  relates  to  systems 
engineering.  Specifically,  the  site  breaks  down  each  of  the  systems  engineering 
processes  and  details  how  to  conduct  each  process  with  respect  to  the  human 
element.  The  overall  site  map  is  provided  as  Figure  8. 
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Figure  8.  ONR  S&T  Manning  Affordability  site  map  (from 

http://www.manninqaffordability.com/s&tweb/index.htm) 


7.  Ongoing  Research 

Research  is  underway  by  the  National  Academy  of  Sciences  to  develop  a 
study  on  HSI  resources  (John  Lockett,  personal  communication,  Sep  29,  2006). 
The  Human-Systems  Design  Support  for  Changing  Technology  study  panel  was 
chartered  to  “develop  a  vision  for  incorporating  human  factors  engineering 
considerations  into  the  design  process  for  complex  systems,  especially  in  view  of 


24 


technologies  that  are  changing  rapidly  and  increasing  in  complexity”  (The 
National  Academies,  2006).  One  of  the  panel’s  goals  is  to  issue  a  report  which 
considers  “the  techniques  for  an  integrated,  interdisciplinary,  adaptable  human- 
system  design  methodology  and  tools  that  can  be  applied  in  both  civilian  and 
military  arenas”  (The  National  Academies,  2006).  The  study  initiated  in  2005,  and 
while  much  effort  has  been  devoted  to  this  endeavor,  the  group’s  findings,  to 
include  databases  and  reports,  are  not  expected  to  be  available  until  spring  of 
2007. 

8.  An  Explanatory  Note 

It  is  apparent  that,  as  with  most  complex  systems,  there  is  no  global 
solution  to  the  question  of  what  resource,  or  set  of  resources,  will  be  most 
effective.  Any  attempt  to  identify  the  ‘essential’  resources  for  the  evaluation  of 
HSI  will  inevitably  limit  the  ability  to  properly  evaluate  a  system.  Each  system’s 
intricate  design  requires  careful  study  by  properly  educated  and  experienced 
personnel  to  most  accurately  assess  the  level  of  integration  a  system  has 
achieved  across  the  HSI  domains.  Defense  Acquisition  University  identifies 
these  individuals  as  HSI  Analysts  (USD/(AT&L),  2006).  More  will  be  discussed 
with  respect  to  these  individuals  in  Chapter  V.  Carr  and  Scholl  (2006)  describe 
the  HSI  process  as  a  refining  process  whereby  the  relevant  domains  are  placed 
under  scrutiny  to  ensure  that  every  possible  advantage  is  identified  and  weighed. 
The  products  of  this  process  are  referred  to  as  the  ‘silver  bullets’  of  HSI.  It  is 
important  to  note  that  the  term  is  used  in  the  plural  form.  Each  system’s  unique 
design,  characteristics  and  requirements  will  require  an  equally  unique  ‘silver 
bullet’. 

The  database  described  in  this  chapter  presents  the  HSI  analyst  with  a 
starting  point  when  planning  HSI  evaluations.  The  purpose  for  these  evaluations, 
as  well  as  how  HSI  fits  into  Defense  Acquisition  are  the  subject  of  the  next 
chapter. 
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III.  THE  LANGUAGE  OF  HSI  AND  ACQUISITION 


A.  DEFENSE  ACQUISITION 

As  previously  noted,  a  key  area  of  influence  for  Human  Systems 
Integration  is  in  the  design,  development,  manufacturing  and  support  of 
processes  and  products.  The  Department  of  Defense  (DoD)  utilizes  the  Defense 
Acquisition  Management  Framework  as  the  means  for  procuring  these  new 
items,  as  well  as  for  improving  existing  processes  and  products  (Figure  9).  Using 
the  framework,  the  DoD  identifies  each  product  or  process  as  a  program,  and 
assigns  a  program  manager  to  oversee  the  procurement  of  the  item.  Program 
managers,  with  the  support  of  Integrated  Product  Team  (IPT)  members,  make 
the  decisions  affecting  a  program  based  on  information  about  the  program’s 
budget,  time  constraints,  and  key  performance  parameters.  Since  this 
information  is  vital  in  decision-making,  the  chief  concerns  of  program 
management  center  on  the  cost,  schedule  and  performance  of  a  program 
mitigated  by  the  associated  risks.  In  fact,  DoD  acquisition  professionals  are 
trained  on  the  evaluation  of  these  three  areas  as  they  relate  to  a  given  program 
(USD/(AT&L),  2001).  Often,  the  success  of  a  program  manager  is  measured  by 
how  well  he/she  manages  the  cost,  schedule,  and  performance  of  the  program. 

User  Needs  fit 
Technology  Opportunities 


/  \  \ 

A  A  c  IOC  FOC 
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Figure  9.  Defense  Acquisition  Framework  (From 
http://akss.dau.mil/dag/DoD5002/Figure1  .asp) 


The  costs  associated  with  any  system  are  not  restricted  to  the  amount  of 

money  spent  to  buy  the  item.  In  order  to  better  understand  how  much  resources 
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will  be  required  by  a  system  over  its  entire  life,  a  metric  is  needed  which  takes 
into  account  the  cost  of  designing,  developing  and  manufacturing  the  system,  as 
well  as  the  necessary  operational  and  sustainment  costs,  and  the  eventual  cost 
of  disposal.  Life  Cycle  Cost  (LCC)  is  the  term  used  to  describe  the  summation  of 
all  costs  associated  with  the  system.  There  is  sufficient  anecdotal  evidence  that 
Program  Managers  have,  on  occasion,  made  program  decisions  based  on  the 
short-term  acquisition  cost  of  a  system  without  paying  attention  to  the  associated 
future  costs  that  would  be  required.  In  these  situations,  cost  overruns  are 
inevitable.  By  determining  the  LCC  of  a  system,  Program  Managers  and  the 
DoD  can  conduct  fiscal  management  with  more  accurate  detail,  avoiding  the 
‘hidden  costs’  that  may  otherwise  surface.  For  clarification,  the  costs  described 
in  the  remainder  of  this  study  relate  to  the  system’s  LCC. 

To  assist  program  managers  with  keeping  programs  on  schedule,  within 
budgetary  constraints,  and  meeting  performance  parameters,  the  DoD  provides  a 
host  of  courses,  guidelines,  best  practices  and  additional  information  available 
through  the  Defense  Acquisition  University  (DAU).  Managing  the  trade-offs 
between  the  triple  constraints  of  cost,  schedule  and  performance  is  similar  to  the 
management  of  the  trade-offs  between  domains  of  HSI. 

The  DAU  has  implemented  specific  guidance  with  respect  to  the 
relationships  between  the  three  key  areas  of  interest  to  the  program  manager. 
The  DAU  instruction  describes  the  interrelatedness  of  cost  and  schedule  through 
the  concept  of  Earned  Value  Management  (EVM)  and  the  associated  EVM 
system  (Berta  &  Mandley,  2005).  The  EVM  system  allows  program  managers  to 
measure  the  amount  of  work  going  into  a  project  and  the  productivity  of  that  work 
in  dollars  and  hours.  While  the  EVM  system  provides  excellent  information  on 
cost  and  schedule,  system  performance,  measured  by  the  effectiveness  of  the 
system  in  meeting  the  key  performance  parameters,  is  tracked  only  in  the  early 
stages,  or  as  changes  to  the  system  require.  Since  the  human  element  of  a 
program  offers  the  greatest  potential  for  improving  system  performance  (Booher, 
2002),  incorporating  HSI  into  the  acquisition  process  gives  program  managers 
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up-to-date  information  on  the  system’s  performance.  Acknowledging  this 
opportunity,  the  DoD  includes  HSI  in  the  latest  versions  of  its  acquisition-related 
documents. 

B.  HSI  IN  DEFENSE  ACQUISITION 

The  main  sources  of  information  on  acquisition  in  the  Department  of 
Defense  are  the  DoD  5000  series  documents  and  the  Defense  Acquisition 
Guidebook  (DAG).  Within  the  5000.1  directive,  HSI  is  implied  or  mentioned  by 
name  in  the  enclosure  under  the  subsections  of  Safety  and  Total  Systems 
Approach.  Specifically,  the  document  directs  that  program  managers  “shall  apply 
Human  Systems  Integration  to  optimize  total  system  performance,  operational 
effectiveness,  and  suitability,  survivability,  safety,  and  affordability”  (USD/(AT&L), 
2002a).  After  mentioning  HSI  as  one  of  the  purposes  of  the  System 
Development  and  Demonstration  phase  in  the  acquisition  framework,  DoD 
Instruction  5000.2  includes  an  entire  enclosure  on  the  implementation  of  HSI. 
The  enclosure  opens  with  a  statement  that  the  program  manager  “shall  have  a 
comprehensive  plan  for  HSI  in  place  early  in  the  acquisition  process  to  optimize 
total  system  performance,  minimize  total  ownership  costs,  and  ensure  that  the 
system  is  built  to  accommodate  the  characteristics  of  the  user  population  that  will 
...[use]...  the  system”  (USD/(AT&L),  2002b).  Unfortunately,  no  additional 
instruction  pertaining  to  the  HSI  plan  is  given,  and  the  remainder  of  the  enclosure 
describes  typical  activities  that  should  be  conducted  in  each  of  the  individual 
domains.  In  effect,  the  instruction  reduces  HSI  to  a  grouping  together  and 
managing  of  activities  related  to  the  human.  The  latest  draft  version  of  the 
Acquisition  Strategy  Guide  states  that  “the  Program  Manager  should  pursue  HSI 
initiatives  within  the  strategy  to  optimize  total  system  performance  and  minimize 
[total  ownership  cost]”  (Defense  Acquisition  University  Press,  2003).  However, 
nowhere  does  the  DoD  require  mandatory  briefings  of  HSI  to  senior  leadership  at 
the  major  decision  points  of  a  program  known  as  the  Milestone  briefings. 

However,  HSI  is  enjoying  increased  visibility  in  the  acquisition  community. 

The  latest  version  of  the  Defense  Acquisition  Guidebook  (DAG)  devotes  an  entire 
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chapter  to  HSI  (USD/(AT&L),  2006).  Building  upon  the  information  given  in  the 
Directive  and  Instruction,  chapter  6  of  the  DAG  provides  a  more  detailed 
description  of  the  activities  associated  with  each  domain  of  HSI,  as  well  as 
detailing  the  need  for  integration  of  the  efforts  occurring  in  each  of  the  domains. 
Specifically,  the  DAG  recognizes  integration  as  the  “key  to  a  successful  HSI 
strategy”.  The  US  Navy’s  NAVAIR  Acquisition  Guide  describes  the  importance 
of  HSI  in  acquisition,  with  the  goal  “to  influence  system  design  and  associated 
support  requirements  so  that  ...systems  can  be  operated  and  maintained  in  the 
most  cost-effective  and  safe  manner  consistent  with  manpower  structure, 
personnel  aptitude  and  skill,  and  training  resource  constraints”  (NAVAIR,  2004). 

Any  attempt  to  cohesively  incorporate  the  efforts  of  HSI  evaluations  and 
the  acquisition  management  strategy  currently  supported  by  the  DoD  must  speak 
in  the  language  of  defense  acquisition — that  is,  cost,  schedule,  and  performance. 
Expressing  HSI  in  these  terms  requires  identifying  the  parameters  to  be 
measured  and  defining  the  relationship  that  each  parameter  has  with  program 
costs,  schedule,  and  performance.  For  each  program,  this  process  is  unique. 
The  model  described  in  Chapter  IV  provides  a  workspace  for  HSI  evaluation — 
converted  to  the  language  of  defense  acquisition — without  restricting  the 
evaluator.  With  myriad  parameters  for  evaluation,  an  example  of  the  conversion 
of  a  simple  parameter  with  respect  to  each  domain  may  prove  useful.  The  next 
section  describes  how  each  domain  may  be  expressed  in  terms  of  one  of  the 
most  notable  dialects  of  defense  acquisition:  life-cycle  costs. 

C.  AN  EXAMPLE:  EXPRESSING  THE  DOMAINS  OF  HSI  IN  TERMS  OF 

LIFE-CYCLE  COSTS 

Cost  estimating  is  a  vital  part  of  defense  acquisition.  In  order  to  produce  a 
preliminary  budget,  the  costs  involved  must  be  estimated.  Thereafter,  any 
potential  change  to  the  system  must  be  evaluated  with  respect  to  its  effect  on  the 
program’s  budget.  The  resulting  estimates  are  only  as  accurate  as  the  amount  of 
variability  accounted  for  by  the  cost  estimator.  Essentially,  cost  estimators 
attempt  to  define  as  much  of  the  trade-space  surrounding  a  decision  as  possible. 
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For  this  reason,  a  more  accurate  estimation  of  life  cycle  cost  is  achieved  through 
understanding  how  changes  affect  the  human  element  of  the  system.  Since  HSI 
in  the  US  Navy’s  SEAPRINT  is  currently  envisioned  by  eight  domains,  the 
possible  conversion  of  each  of  these  domains  into  life-cycle  costs  is  provided 
below  as  an  example. 

1.  Manpower 

By  definition,  manpower  relates  solely  to  the  number  of  individuals 
required  to  operate  and  maintain  a  system  (U.S.  Navy,  2006).  To  put  this  in 
terms  of  cost  we  must  consider  all  expenses  associated  with  manpower. 
Expenses  related  to  military  manpower  include  wages,  services  such  as  health 
care,  housing,  sustenance,  transportation,  etc.  Once  these  expenses  are 
identified,  a  per-individual  amount  can  be  derived  which  will  provide  a  unit  cost 
for  unit  of  manpower.  In  essence,  knowing  that  a  system  will  require  twenty 
individuals  to  operate  it,  a  rough  estimate  of  the  cost  to  supply  those  twenty 
individuals  can  be  determined. 

2.  Personnel 

Returning  to  the  definitions  of  HSI  domains,  the  personnel  domain  is 
associated  with  the  cognitive  and  physical  capabilities,  as  well  as  additional 
human  characteristics  and  individual  experiences  necessary  for  a  system  to 
function  at  a  given  level  (U.S.  Navy,  2006).  Costs  associated  with  personnel 
center  chiefly  on  the  selection  process  and  assignment  of  compensation. 
Determination  of  the  level  of  individual  attributes,  or  perhaps  the  scarcity  of  the 
personnel  attributes  required,  will  allow  for  identification  of  the  requisite  wage  for 
such  individuals.  The  cost  of  personnel  can  therefore  be  described  in 
compensation  (easily  converted  to  dollars)  per  individual  required. 

3.  Training 

Current  standard  business  practices  call  for  an  accounting  of  costs 
incurred  by  a  company  and  benefits  derived  from  those  costs.  The  ratio  of  these 
two  amounts,  known  as  Return  on  Investment  (ROI),  is  commonly  used  to 
evaluate  training  programs  (Bartel,  2000).  Total  costs  of  training  include 
development  costs,  lost  time  from  performing  work,  overhead  of  education 


31 


department,  physical  materials,  etc.  While  instructor  based  training  may  incur 
higher  costs  such  as  travel  to  and  from  training,  instructor  salary,  and  even 
refreshments,  computer-based  training  also  incurs  costs  stemming  from  media 
programming,  production  and  distribution  (Connor,  2002). 

4.  Human  Factors 

The  field  of  human  factors  has  used  cost  as  an  evaluative  measure  of 
usability  for  many  years.  The  idea  of  a  curvilinear  relationship  between  dollars 
spent  on  usability  and  benefit  received  from  such  efforts  is  the  basis  for  cost- 
justifying  usability  (Bias  and  Mayhew,  1994).  There  is  a  point  where  increased 
spending  or  effort  results  in  diminishing  returns. 

As  with  most  engineering  efforts,  the  human  factors  engineering  industry 
estimates  a  certain  amount  of  production  for  each  hour  worked.  With  respect  to 
cost,  human  factors  can  be  described  as  the  level  of  effort — defined  as  hours 
billed — given  to  the  human  factors  design  of  a  system  throughout  the 
development  process.  The  cost  per  hour  can  be  calculated  and,  when  multiplied 
by  the  number  of  hours  required,  will  result  in  a  cost  for  the  human  factors  effort. 
In  theory,  the  more  hours  dedicated  to  human  factors  design,  the  more  usable 
the  system  will  be. 

5.  Safety 

Describing  system  safety  as  a  cost  measure  requires  an  evaluation  of  the 
potential  number  of  hours  lost  due  to  mishaps  and  converting  those  lost  hours  to 
a  dollar  amount  by  multiplying  the  number  of  hours  by  the  per  hour  wage  for  the 
affected  individual(s).  This  would  result  in  the  ability  to  ascertain  the  associated 
costs  given  the  level  of  safety  designed  into  a  system.  Increasing  the  level  of 
safety  would  reduce  the  associated  costs  just  as  lowering  the  level  of  safety 
would  increase  associated  costs. 

6.  Health  Hazards 

Within  a  system,  each  system-caused  stressor  adds  to  the  level  of  hazard 
inherent  in  the  operation  of  the  system.  Mitigation  of  these  hazards  reduces  the 
risk  of  bodily  injury  or  death.  Stress  fractures  caused  by  repeated  jumps  with 
+200  pound  packs,  inadequate  design  of  steam  pipes  in  a  naval  vessel,  and  high 
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levels  of  vibration  from  sustained  operations  in  a  tank  can  all  lead  to  injuries 
ranging  from  mild  to  severe  and  resulting  in  the  temporary  or  permanent  loss  of 
personnel.  Somewhat  similar  to  the  costs  of  safety  described  as  hours  lost,  the 
costs  of  health  hazards  can  be  calculated  by  assessing  the  estimated  associated 
treatment  costs  for  level  of  exposure  allowed  in  the  design  of  a  system.  Added  to 
this  are  the  hours  lost  by  such  injuries,  and — in  the  case  of  permanent 
disability — the  costs  of  compensating  the  injured  personnel  and  costs  of 
acquiring  another  equally  qualified  individual.  Exposure  to  nuclear,  biological, 
and  chemical  weapons  will  have  increased  costs  as  the  level  of  protection 
diminishes.  As  more  attention  is  paid  to  addressing  the  health  hazards  of  the 
system,  the  associated  costs  to  the  system  can  be  reduced. 

The  interrelatedness  of  the  HSI  domains  is  readily  apparent  from  this 
example.  Improved  efforts  in  the  area  of  health  hazards  and  human  factors  can 
reduce  costs  associated  with  training,  manpower,  and  safety. 

7.  Human  Survivability 

Survivability  describes  the  ability  of  the  human  in  the  system  to  remain 
unharmed  by  non-system-related  causes  throughout  operation  of  the  system 
(U.S.  NPS,  2006).  The  level  of  survivability  afforded  by  the  system  will  determine 
the  number  and  frequency  of  personnel  being  gained  by  the  system  (through  loss 
of  personnel),  as  well  as  injury  rate  and  severity  from  outside  causes.  Costs 
associated  with  survivability  should  therefore  include  a  combination  of  the 
predicted  probability  of  kill,  a  measurement  of  possible  injury  rate,  and  costs  for 
acquiring  additional  adequate  personnel  and  costs  associated  with  injury 
recovery. 

8.  Habitability 

Arguably  the  most  difficult  of  the  domains  to  define,  habitability  refers  to 
accounting  for  the  physical  and  psychological  needs  associated  with  sustained 
operation  of  the  system  (Naval  Postgraduate  School,  2006).  State-of-the-art 
berthing,  with  soundproof  sleep  chambers  and  a  schedule  designed  to  minimize 
fatigue  during  the  planned  year-long  mission,  may  rank  low  in  habitability  if  there 
exist  no  means  for  personnel  to  communicate  with  loved  ones  or  enjoy  down- 
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time.  The  cost  of  habitability  can  be  expressed  as  the  costs  associated  with 
maintaining  adequate  services  and  facilities  to  induce  personnel  to  remain  as 
part  of  the  system. 


D.  THE  NEXT  STEP 

While  the  exact  dimensions  of  the  HSI  foundation  are  still  being 
deliberated,  it  is  readily  apparent  that  the  world  of  defense  acquisition  rests 
solidly  within  its  borders.  When  discussing  metrics  for  acquisition,  cost,  schedule, 
performance,  and  risk  are  the  essential  dependent  variables.  Determining  how 
acquisition  program  decisions  will  affect  the  cost,  schedule  or  performance  of  the 
system  allows  the  analyst  to  communicate  most  effectively  with  program 
management. 

Each  defense  acquisition  system  will  require  a  tailored  approach  to  HSI. 
The  temptation  to  produce  a  single  equation  which  could  be  applied  to  any 
acquisition  program  will  not  be  satisfactory.  However,  since  the  concept  of  HSI 
is  based  on  the  evaluation  of  trade-offs  and  interactions  between  parameters,  a 
general  process  and  workspace  within  which  an  individual  evaluation  can  be 
developed  is  critical,  and  will  provide  much  needed  guidance  for  those  individuals 
conducting  HSI  evaluations.  The  next  chapter  presents  an  approach  to  the 
design  of  such  a  workspace. 
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IV.  AN  INTERACTIVE  HSI  VISUALIZATION  TOOL  FOR  THE 
ACQUSITION  PROFESSIONAL 

A.  VISUALIZING  HSI 

A  logical  process  for  the  Human  Systems  Integration  evaluation  of  an 
acquisition  program  can  be  summarized  in  four  steps:  planning  and  conducting 
the  evaluation,  compiling  the  results,  presenting  the  findings,  and  receiving 
feedback  from  program  decision-makers.  Using  this  process  as  a  guide,  a  series 
of  interfaces  were  designed  to  aid  both  the  individuals  conducting  the  HSI 
evaluations  and  those  who  would  use  the  results  in  making  changes  to  the 
program.  The  interfaces  were  designed  with  the  intent  to  be  quickly  and  easily 
interpreted.  To  this  end,  they  are  highly  visual  in  nature.  The  three  interfaces 
include  an  HSI  Resource  Search  interface,  a  parameter  interaction  editor,  and 
the  culminating  HSI  Trade  Space  Tool  (HSI  TST).  Each  of  the  three  interfaces  is 
described  in  detail  below. 

B.  PLANNING  AND  CONDUCTING  THE  EVALUATION 

In  order  for  any  acquisition  program  to  be  initiated,  the  need  for  which  it  is 
being  developed  must  be  defined.  As  part  of  this  process,  the  Initial  Capabilities 
Document  (ICD)  details  “the  need  for  a  materiel  approach... to  satisfy  specific 
capability  gap(s)”  (DAU,  2006a).  When  necessary,  the  ICD  leads  to  production 
of  the  Capability  Development  Document  (CDD).  The  CDD  further  defines  the 
necessary  operational  mission  performance  parameters.  Based  on  the  human 
performance  requirements  identified  in  the  program’s  CDD,  the  HSI  analyst  must 
develop  a  plan  for  conducting  the  necessary  assessments  to  ensure  the  program 
meets  performance  parameters. 

Similar  to  the  HSI  Resource  Data  Collection  Sheet  described  in  Chapter  II, 
the  user  interface  designed  for  planning  an  HSI  evaluation  (Figure  10)  provides 
key  information  with  respect  to  the  myriad  products  available  for  use  in 
evaluating  specific  human  performance  parameters.  Whereas  the  data  collection 
sheet  allows  for  the  input  of  data,  the  interface  allows  the  user  to  search  the 
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database  of  resources  by  a  variety  of  fields:  name,  classification,  metric,  and  HSI 
domain.  After  filling  in  the  relevant  data  and  executing  the  search  function,  the 
analyst  is  presented  with  a  list  of  resources  matching  the  query.  Selecting  any  of 
the  listed  resources  opens  the  database  information  on  that  resource,  allowing 
the  analyst  to  determine  the  best  resource  for  the  requirement.  Once  methods 
for  assessing  the  human  performance  requirements  have  been  identified,  the 
HSI  analyst  can  proceed  with  conducting  the  evaluation  knowing  all  requirements 
are  accounted  for. 


Figure  10.  HSI  Resource  Search  Interface 

C.  COMPILING  THE  RESULTS 

Once  an  analysis  of  the  design  decisions  has  been  conducted  on  all 
relevant  parameters,  the  HSI  analyst  must  create  a  logical  relationship  between 
the  parameters.  It  is  at  this  point  that  true  Human  Systems  Integration  takes 
place.  By  identifying  the  relationships  between  parameters,  the  impact  of  design 
changes  can  be  made  much  clearer. 

Considering  the  possible  parameters  for  evaluation,  an  HSI  analyst  may 
be  required  to  define  a  vast  number  of  relationships  in  order  to  account  for 
enough  of  the  trade  space  to  provide  a  usable  solution.  Looking  solely  at  the 
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two-way  interactions  between  parameters  could  severely  limit  the  effectiveness 
of  the  evaluation.  Taking  the  historical  conceptualization  of  HSI  as  seven  to  nine 
individual  domains,  the  complete  list  of  two-way  interactions  only  accounts  for  as 
much  as  17.5%  or  as  little  as  7%  of  the  possible  interactions.  It  would  be 
irresponsible  to  base  decisions  affecting  large-scale  programs  on  such  limited 
assumptions.  Expanding  the  model  to  include  all  possible  interactions  (between 
128  and  502),  however,  would  result  in  an  excessive  load  for  the  analyst  in  both 
time  and  complexity.  In  order  to  limit  the  scope  of  the  model,  a  fresh  approach  to 
the  traditional  domains  of  HSI  was  considered. 


Cameron  and  Rench  (2005)  propose  that  HSI  consists  of  experts  in  the 
four  domains  of  Manpower,  Personnel,  Training,  and  Human  Factors,  working 
toward  goals  related  to  the  remaining  domains.  These  four  key  domains  exist  in 
each  of  the  services’  HSI  paradigms,  with  the  services  differing  in  the  observance 
of  the  final  three  to  five  domains  (Figure  1 1).  Expanding  on  the  idea  posited  by 
Cameron  and  Rench,  the  four  key  domains  can  be  viewed  as  inputs  to  the 
system  of  HSI  evaluation,  with  the  remaining  domains  viewed  as  functions  of  the 
interactions  between  key  domains,  and  the  output  being  total  system 
performance.  Focusing  on  these  four  domains  as  the  driving  force  behind  HSI 
effectiveness  and  representing  the  remaining  traditional  domains  as  functions — 
along  with  the  acquisition  imperatives  of  cost,  schedule,  performance  and  risk — 
reduces  the  workload  to  a  more  manageable  level:  defining  27  interactions  and 
four  functions. 


HSI  Domains  other  than  Manpower,  Personnel,  Training,  and  Human  Factors 


U  S  ARMY 

•Soldier  Survivability 
•System  Safety 
•Health  Hazards 


U  S.  NAVY 
•Human  Survivability 
•System  Safety 
•Health  Hazards 
•Habitability 


U  S.  AIR  FORCE 

•Human  Survivability 

•System  Safety 

•Occupational 
Health  Hazards 

•Habitability 

•Environment 


Figure  1 1 . 


Service  specific  HSI  domains 
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By  scoping  the  model  to  the  four  key  HSI  domains  and  their  interactions 
with  acquisition  parameters,  the  design  of  the  interface  becomes  more  tailored  to 
the  Defense  Acquisition  HSI  analyst.  The  field  of  HSI  is  still  evolving,  and  may 
not  be  fully  defined.  Ensuring  customizability  of  the  interface  would  allow  HSI 
professionals  to  experiment  with  optional  parameters.  Also,  it  would  allow  the 
interface  to  evolve  with  the  current  understanding  of  HSI.  To  this  end,  the 
interface  was  designed  with  preset  parameters  for  each  of  the  services  as  well  as 
a  customizable  feature  where  the  user  could  create  and  define  each  parameter. 

A  B 


After  selecting  a  parameter  to  edit  from  a  dropdown  menu  in  the  ‘Edit 
Parameter’  box  (see  A  in  Figure  12) — or  selecting  to  enter  a  new  parameter — the 
HSI  analyst  enters  information  defining  the  parameter.  This  information  includes 
editing  the  parameter  name,  the  unit  of  measure,  and  a  series  of  inputs 
describing  the  scale  to  be  displayed.  The  interface  was  designed  to  allow  the 
analyst  to  enter  ranges  of  acceptable,  questionable,  and  unacceptable  levels 
associated  with  the  acquisition  parameters  of  cost,  schedule,  performance  and 
risk.  These  ranges  were  designed  to  be  used  in  the  decision-maker’s  interface 
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described  later.  The  information  for  each  parameter  is  saved  by  selecting  the 
‘Save’  button  at  the  bottom  of  the  ‘Edit  Parameter’  box.  Additionally,  options  for 
clearing  the  contents  or  deleting  a  parameter  are  offered. 

Once  this  process  has  been  completed  for  each  parameter,  the  HSI 
analyst  defines  the  interactions  between  parameters  through  the  ‘Edit  Interaction’ 
box  (see  B  in  Figure  12).  Here,  the  HSI  analyst  can  manipulate  the  two-way 
interactions  through  selecting  the  appropriate  parameters  and  using  one  of  three 
methods  to  describe  the  relationship:  text  input,  imported  data,  and  line 
manipulation.  The  text  input  option  allows  the  HSI  analyst  to  enter  a 
mathematical  equation  which  the  interface  then  displays  graphically.  If  the 
analyst  has  previously  correlated  the  two  parameters — for  example,  identifying 
the  costs  associated  with  certain  manpower  levels — the  data  can  be  imported 
directly  into  the  interface  by  selecting  the  ‘Import  Data’  button.  The  user  is  then 
prompted  to  identify  the  appropriate  file  which  contains  the  XY  coordinates.  The 
coordinates  are  then  displayed  graphically  according  to  the  parameter  settings. 
The  final  option  for  inputting  relational  data  is  to  select  the  general  type  of  line 
from  a  dropdown  menu  resulting  in  a  line  displayed  in  the  graphing  area.  This 
line  can  then  be  manipulated  by  clicking  and  dragging  points  on  the  line.  The 
interface  is  designed  with  the  capability  of  converting  the  manipulated  line  into  an 
equation  which  is  displayed  in  the  equation  text  box.  Once  the  user  is  satisfied 
with  the  result,  the  interaction  is  saved,  and  the  process  continues  for  each 
interaction. 

To  this  point,  the  analyst  has  defined  all  two-way  interactions  between  the 
main  parameters  and  the  acquisition  parameters  of  cost,  schedule,  performance 
and  risk.  Here  it  becomes  difficult  to  graphically  represent  the  final  multi¬ 
parameter  interactions.  The  interface  was  designed  to  input  these  multi¬ 
parameter  interactions  solely  through  the  text  input  option.  When  known,  these 
multi-parameter  interactions  can  greatly  enhance  the  detail  of  the  evaluation. 
Unfortunately,  these  interactions  are  difficult  to  define.  Nevertheless,  the  option 
has  been  designed  into  the  interface.  Potentially  improved  methods  for 

accounting  for  multi-parameter  interactions  are  discussed  in  the  final  chapter. 
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D.  PRESENTING  EVALUATION  RESULTS  AND  RECEIVING  FEEDBACK 

The  main  objectives  of  the  HSI  analyst  are  to  convey  the  level  to  which 
consideration  of  the  human  element  has  been  designed  into  the  system,  and 
demonstrate  the  potential  effects  of  system  changes  on  the  human  element. 
These  objectives  can  be  accomplished  through  a  simple  graphical  representation 
of  the  current  assessment  of  the  system,  as  well  as  the  capability  to  manipulate 
the  current  levels  in  order  to  view  probable  outcomes.  The  HSI  Trade  Space 
Tool  was  designed  with  such  an  interface  in  mind. 

1.  ‘Slider’ concept 

In  2002,  under  a  work  project  for  the  Air  Force’s  311th  Human  Systems 
Wing,  Veridian  Engineering,  with  guidance  from  Air  Force  Major  Robert  Lindberg, 
developed  a  conceptual  tool  that  expressed  the  idea  of  displaying,  in  a  simple 
interface,  the  trade-offs  among  HSI  domains  associated  with  program  decisions 
(Fig  13).  The  intent  of  the  project  was  to  emphasize  the  importance  of 
understanding  the  bivariate  relationships  that  exist  between  the  domains  of  HSI. 
This  project  was  a  simple  demonstration,  and  the  interactions  between  elements 
were  defined  using  curves  derived  from  general  observations. 


Human  Systems  Integration 


More  at  HSI  CoP  . 


HSI  Domains 


Welcome  to  the  HSI  relationships  model.  The  purpose  of  this  model  is  to 
allow  you  to  better  understand  the  relationships  between  each  of  the  HSI 
domains  and  other  key  factors  that  are  essential  for  a  weapon  system. 

Each  of  the  HSI  domains  is  listed  on  the  right  side  of  the  screen.  A  short 
definition  of  the  domain  is  displayed  when  you  hover  your  cursor  over  it.  To 
the  left  of  each  domain  is  a  "slider"  icon.  Notice  that  they  are  all  blue.  The 
active  slider  will  be  orange  (which  means  that  none  are  active  on  the 
Welcome  page!). 

Run  your  cursor  over  the  orange  circles  below.  To  review  the  relationships 
for  a  specific  domain  area  (making  it  active),  click  on  its  corresponding 
orange  button.  You  should  then  notice  that  the  slider  icon  for  the  active 
domain  is  orange.  The  orange  slider  is  the  "independent"  domain  and  is  the 
only  slider  you  will  be  able  to  move.  As  you  move  it  left  or  right,  you  will 
be  able  to  "see"  how  it  impacts  the  other  domains  ("dependent  variables"). 
A  description  is  displayed  by  hovering  over  the  domain  name. 

For  more  information,  click  on  "More  at  HSI  CoP"  located  in  the  top  banner. 
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Figure  13.  HSI  slider  tool  concept  (From  http://Kn.qd- 

ais.com/ASPs/eLearninq/HSI  Sliders/concepts/application. swf) 
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Initially,  the  concept  of  the  model  was  based  on  describing  the  two-way 
relationships  between  a  single  domain  and  each  of  the  others,  as  well  as  the 
relationships  of  that  single  domain  to  the  life  cycle  cost  and  capability  of  the 
system.  As  discussed  previously,  a  more  accurate  model  was  needed  which 
would  include  multivariate  interactions.  While  the  concept  could  remain  relatively 
intact,  the  processes  behind  it,  as  described  in  the  previous  section,  became 
more  involved. 

2.  HSI  Trade  Space  Tool  (HSI  TST) 

The  HSI  TST  interface  is  designed  using  a  concept  similar  to  that  of  the 
slider  tool  developed  for  the  Air  Force.  Each  parameter  is  placed  on  the  left,  with 
descriptive  labels  identifying  the  parameter,  unit  of  measure  and  present  value. 
The  acquisition  parameters  (cost,  schedule,  performance  and  risk)  are  placed  on 
the  right,  with  similar  labels.  Additionally,  the  text  box  associated  with  the 
present  value  for  each  of  the  acquisition  parameters  is  color-coordinated  with  the 
present  value,  based  on  the  ranges  defined  in  the  parameter  editor 
(green=acceptable,  yellow=questionable,  red=unacceptable).  The  capability  to 
select  an  individual  parameter  to  alter  is  accomplished  by  placing  a  selection 
button  above  each  parameter.  Once  selected,  movement  of  the  associated 
slider  bar  presents  the  predicted  changes  in  the  remaining  parameters.  The 
movement  of  the  remaining  parameters  is  designed  to  utilize  the  interactions 
defined  by  the  HSI  analyst,  with  the  acquisition  parameters  defined  as  functions 
of  the  main  parameters.  A  conceptual  design  of  the  HSI  TST  interface  is 
provided  in  Figure  14. 
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Figure  14.  Modified  ‘Slider’ — HSI  Trade  Space  Tool  (TST) 


E.  JOINT  DEVELOPMENT  OF  INTERFACE  SOFTWARE 

At  the  same  time  as  the  interfaces  and  process  described  above  were 
being  developed,  an  effort  to  produce  a  stand-alone  introductory  course  in  HSI 
was  underway  at  the  Naval  Postgraduate  School  by  Professors  Larry  Shattuck 
and  Nita  Lewis  Miller.  They  offered  the  assistance  of  the  Office  of  Continual 
Learning  (OCL)  at  NPS  in  developing  the  software  for  the  interfaces,  with  the 
intent  of  using  the  process  and  interfaces  as  part  of  the  introductory  course.  With 
the  basic  designs  and  detailed  descriptions  of  the  functionality  of  each  interface 
well  defined,  the  OCL  at  NPS  initiated  software  development  of  the  interfaces  in 
late  November,  2006. 

The  design  of  the  various  interfaces  and  coursework  outlined  in  this 
chapter  would  be  of  no  use  if  it  did  not  reach  the  audience  that  can  most 
effectively  use  them.  Identifying  these  individuals,  as  well  as  venues  for 
introducing  the  interfaces  and  coursework  discussed  above,  is  the  purpose  of  the 
next  chapter. 
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V.  DISTRIBUTING  THE  MODEL 


A.  VENUES  OF  DISTRIBUTION 

Recent  guidance  to  acquisition  professionals  identifies  HSI  analysts  as 
those  who  implement  HSI  in  systems  acquisition  and  “assist  program  managers 
by  focusing  attention  on  the  human  part  of  the  system  and  by  integrating  and 
inserting  [domain]  considerations”  (USD/(AT&L),  2006).  However,  the  current 
guidance  does  not  elaborate  on  the  qualifications  of  these  individuals,  how  to  find 
such  individuals,  or  the  process  for  becoming  such  an  individual.  These  HSI 
analysts,  as  well  as  the  acquisition  community  at  large,  are  the  target  group  for 
distribution  of  the  HSI  evaluation  process  and  interfaces. 

Including  HSI  in  acquisition  guidance  opens  the  door  for  regulation  of  the 
qualifications  and  certification  of  those  labeled  HSI  analysts.  Through  the 
Defense  Acquisition  Workforce  Improvement  Act  (DAWIA),  the  Under  Secretary 
of  Defense/Acquisition  Technology  and  Logistics  (USD/AT&L)  outlines  the 
education,  training  and  experience  level  required  for  certification  in  many  of  the 
fields  related  to  defense  acquisition  (i.e.  Program  Management,  Systems 
Engineering,  Contracting,  Test  and  Evaluation).  In  general,  certification  consists 
of  meeting  the  requirements  at  one  of  three  levels.  As  outlined  in  the  Defense 
Acquisition  University  (DAU)  Catalog  Appendix  B  (DAU,  2006),  requirements  for 
certification  follow  the  general  guidelines  listed  below: 

LEVEL  I:  Education —  Baccalaureates  Degree 

Experience — 1  year  experience  in  related  field 
Training —  Basic  acquisition  courses  in  related  field 

LEVEL  II:  Education —  Baccalaureates  Degree  or  Graduate  studies 

Experience — 2  years  experience  in  the  related  field 
Training — Intermediate  acquisition  courses  in  related  field 

LEVEL  III:  Education — Masters  Degree  (Desired) 

Experience —  4  years  experience  in  related  field 

Training — Advanced  acquisition  courses  in  related  field 
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A  similar  certification  process  must  be  included  for  the  HSI  analyst  in  order  to 
ensure  a  more  consistent  level  of  expertise  at  the  HSI  analyst  position.  This 
process  must  be  initiated  by  the  establishment  of  a  functional  career  field  in  HSI 
as  has  been  done  for  all  other  acquisition  related  career  fields. 

It  is  imperative  that  resources  such  as  the  ones  designed  in  the  present 
study  be  available  to  as  many  HSI  analysts  and  acquisition  professionals  as 
possible.  Widespread  availability  will  facilitate  standardization  of  HSI  practices, 
allowing  program  managers  to  know  what  to  expect  in  the  way  of  HSI  analyses. 
To  this  end,  the  key  organizations  and  communities  of  such  individuals  were 
identified  as  points  of  entry  for  the  software.  The  remainder  of  this  chapter 
describes  how  the  model  can  be  introduced  in  the  following  organizations  and 
communities:  Defense  Acquisition  University,  service-specific  acquisition 

communities,  major  defense  contractors,  and  NPS/AFIT/Service  Academies. 

1.  Defense  Acquisition  University 

The  major  provider  of  instructional  material  and  resources  related  to 
acquisition  is  the  Defense  Acquisition  University  (DAU).  The  DAU  provides 
coursework  and  instruction  in  accordance  with  DAWIA.  Under  DAWIA, 
acquisition  professionals  gain  certification,  and  continual  learning,  on  the  best 
practices  of  defense  acquisition. 

Incorporating  the  HSI  evaluation  process  model  and  software  into  DAU’s 

coursework  and  website  is  a  key  step  in  standardization.  As  the  hub  of 

information  for  the  DoD  Acquisition,  Technology,  and  Logistics  (AT&L)  workforce, 

the  DAU  provides  a  clear  path  to  each  of  the  services,  to  industry  professionals, 

and  to  other  federal  agencies.  Simply  placing  the  information  on  the  DAU 

website  will  not  produce  the  desired  results.  Incorporating  the  process  and 

software  into  required  coursework  will  ensure  that  all  acquisition-trained 

personnel  understand — or  at  least  have  been  exposed  to — the  HSI  evaluation 

process.  As  mentioned  in  Chapter  IV,  coursework  is  currently  under 

development  at  the  Naval  Postgraduate  School  (NPS)  where  the  interfaces  and 

process  can  be  inserted.  The  products  of  these  efforts  will  be  a  stand-alone 

introductory  course  on  HSI  as  well  as  a  four-course  certificate  program  in  HSI 
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offered  by  NPS.  Adding  these  courses  to  the  DAU  catalog  and  including  them  as 
part  of  the  basic  acquisition  education  will  disseminate  the  understanding  of  HSI 
throughout  the  fields  of  management,  logistics,  engineering,  and  analysis  which 
receive  training  through  DAU.  While  the  coursework  is  presently  designed  solely 
as  a  certificate  program,  it  may  be  the  answer  to  the  DAU  Level  I,  II,  and  III 
training  requirements  for  certification  as  an  HSI  analyst  described  previously. 

2.  Service  Specific  Acquisition  Communities 

Though  all  of  the  services  rely  on  the  DAU  as  a  source  for  continuing 
education  in  acquisition,  each  service  in  the  DoD  handles  the  acquisition  process 
differently.  Regardless  of  the  composition  of  the  individual  service’s  acquisition 
force,  each  service  educates  its  acquisition  personnel  on  the  service  specific 
processes  involved  with  the  acquisition  process.  This  service  specific  education 
includes  a  basic  understanding  of  systems  thinking  as  it  relates  to  program 
management.  As  has  been  argued  throughout  this  study,  recognizing  the  human 
as  a  key  component  in  the  system — and  the  component  where,  through  proper 
utilization,  the  most  benefit  can  be  gained — is  vital  to  systems  acquisition.  The 
products  of  this  study  can  provide  the  means  whereby  this  understanding  can  be 
incorporated  into  the  service  specific  acquisition  management  processes. 

3.  Major  Defense  Contractors 

Most  of  the  work  involved  with  the  acquisition  of  a  system  is  conducted  by 
defense  contractors.  Knowing  that  the  more  comprehensive  understanding  they 
have  of  the  acquisition  process  makes  them  more  appealing  in  the  selection 
process,  defense  contractors  attempt  to  stay  abreast  of  the  changing  regulations 
and  requirements  set  forth  in  defense  acquisition  guidance.  In  2002,  both  Boeing 
and  Lockheed-Martin  entered  into  strategic  partnerships  with  DAU  in  order  to 
ensure  best  practices  in  acquisition  management  (Glass,  2002a;  Glass,  2002b). 
This  strategic  partnership  provides  contractor  personnel  the  same  opportunity  for 
instruction  as  any  other  defense  personnel.  Those  defense  contractors  who 
have  created  these  ties  to  DAU  will  benefit  from  the  coursework  outlined  above. 
However,  not  all  defense  contractors  have  made  these  ties.  Where  necessary, 
providing  these  defense  contractors  with  the  process,  software  and  instruction  on 
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HSI  will  afford  them  the  opportunity  to  remain  competitive  in  the  selection 
process  and  ensure  a  higher  caliber  of  proposals  where  HSI  is  a  concern. 

4.  Service  Academies,  NPS,  and  AFIT 

While  the  greatest  visibility  of  HSI  may  lie  within  the  acquisition 
community,  the  concept  of  Human  Systems  Integration  relates  to  many  other 
academic  degrees  sought  after  by  DoD  personnel,  beginning  with  the  service 
academies.  Coursework  in  HSI  fits  easily  into  academic  departments  of  the  U.S. 
Air  Force,  Naval,  Coast  Guard  and  Military  Academies.  Both  the  U.S.  Military 
Academy  and  the  U.S.  Air  Force  Academy  have  Behavioral  Sciences  and 
Leadership  departments.  Additionally,  each  of  the  four  service  academies  has  a 
department  or  degree  program  related  to  systems  engineering:  Systems 
Engineering  Department  (USMA),  Systems  Engineering  &  Systems  Engineering 
Management  degree  program  (USAFA),  Department  of  Weapons  and  Systems 
Engineering  (USNA),  Engineering  and  Management  Departments  (USCGA). 
Providing  an  understanding  of  HSI  through  introduction  of  the  software  and 
process  described  in  Chapter  IV  at  the  undergraduate  level  will  instill  the 
importance  of  systems  thinking — especially  related  to  the  human — that  will  drive 
the  future  of  HSI.  While  not  all  academy  graduates  will  seek  higher  degrees  in 
HSI  or  Systems  Engineering,  they  will  certainly  be  involved  in  human-systems 
and/or  with  HSI  at  some  point  in  their  careers.  Their  previous  exposure  to  HSI 
could  tip  the  scales  toward  a  human-centered  decision. 

Academy  graduates  who  desire  to  further  explore  the  field  of  HSI  should 
find  the  opportunity  to  do  so  within  the  military  academic  community.  As 
mentioned  in  Chapter  I,  the  Naval  Postgraduate  School  offers  a  Masters  degree 
in  Human  Systems  Integration.  However,  similar  to  the  undergraduate  level 
programs,  HSI  can  be  easily  integrated  into  other  programs  at  NPS,  such 
as  the  Systems  Engineering  department,  as  well  as  through  the  Air  Force 
Institute  of  Technology’s  Department  of  Systems  and  Engineering  Management 
and  Center  for  Systems  Engineering. 
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5.  Result  of  Proper  Distribution 

Incorporating  the  process  and  software  as  outlined  above  creates  an 
unbroken  link  of  HSI  understanding  through  the  entire  acquisition  community  and 
the  acquisition  process.  The  potential  exists  to  educate  all  essential  personnel  in 
the  significance  of  Human  Systems  Integration  and  the  potential  improvement 
that  focusing  on  the  human  can  produce.  From  senior  leadership  who  determine 
new  capabilities  are  required,  to  the  users  identified  to  participate  in  initial 
capabilities  studies,  to  the  acquisition  professionals  who  plan  for  and  execute  the 
management  of  the  acquisition,  to  the  contractors  who  are  selected  to  deliver  the 
end  produce,  a  clearer  understanding  of  HSI  will  result  in  better  systems. 
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VI.  FUTURE  RESEARCH  AND  SUMMARY 


A.  FUTURE  RESEARCH 

While  the  present  study  has  described  interfaces  to  aid  HSI  analysts  and 
acquisition  program  decision-makers  in  communicating  human  effectiveness  in 
the  system,  continued  research  should  be  conducted  to  ensure  that  this 
approach  provides  the  most  accurate  description  of  the  parameter  interactions. 
Additional  approaches  for  considering  multivariate  HSI  trade  offs  have  been 
suggested.  These  approaches  include  nearly  orthogonal  Latin  hypercubes,  and 
optimal  computing  budget  allocation. 

One  approach  to  modeling  HSI  involves  the  use  of  nearly  orthogonal  Latin 
hypercubes  (Lucas  &  Sanchez,  2003).  Through  models  based  on  this  concept, 
the  HSI  analyst  would  conduct  evaluations  on  parameters  similar  to  the  process 
presented  in  this  study.  With  the  results  of  these  evaluations,  and  after  defining 
the  parameters  for  the  model,  the  HSI  analyst  would  be  presented  with  a  limited 
series  of  scenarios — selected  through  application  of  nearly  orthogonal  Latin 
hypercubes.  In  this  manner,  the  HSI  analyst  would  not  need  to  solve  every 
possible  scenario.  The  limited  series  would  provide  sufficient  statistical  data  from 
which  any  scenario  could  be  evaluated. 

Another  approach,  which  similarly  attempts  to  effectively  reduce  the 
enormity  of  scenarios  involved  in  multivariate  analyses,  is  optimal  computing 
budget  allocation  (OCBA)  for  simulation  (Chen,  H.  C.,  Lin,  J.,  Yucesan,  E.  & 
Chick,  S.  E.,  2000).  Through  this  approach,  a  highly  efficient  number  of 
scenarios  are  determined  with  the  intent  to  significantly  reduce  total  costs 
associated  with  simulation. 

The  present  models  of  HSI  must  be  refined  and  validated.  Researchers 
must  conduct  detailed  investigations  into  the  interactions  which  exist  among  the 
domains  of  HSI — whatever  those  domains  may  end  up  being.  Having  a  better 
concept  of  how  the  domains  interact  will  reduce  the  uncertainty  with  which  HSI 
analysts  are  forced  to  deal  with  at  present. 
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In  the  same  manner  as  basic  use  case  data  were  collected  for  fields  such 
as  cost  estimation  and  usability,  HSI  professionals  must  begin  collecting  and 
consolidating  use  case  information  on  the  outcomes  of  program  decisions  as 
they  affect  the  human  element.  Building  a  database  of  HSI  domain-assessed 
systems  will  provide  future  analysts  with  vital  information  as  they  attempt  to 
influence  program  decisions  in  the  earliest  stages  of  system  design.  Similarly, 
building  a  database  of  the  observed  interactions  between  HSI  system 
parameters  will  result  in  a  knowledge  base  from  which  HSI  analysts  can  draw 
when  identifying  human  performance  parameters  in  future  system  capabilities. 

B.  SUMMARY 

The  concept  of  Human  Systems  Integration  is  being  included  increasingly 
in  the  world  of  Department  of  Defense  acquisition.  As  that  role  continues  to  be 
defined,  it  is  imperative  that  processes  for  conducting  HSI  evaluations  be 
standardized  across  the  DoD.  Such  standardization  removes  the  ambiguity 
surrounding  HSI  and  allows  decision-makers  to  know  what  to  expect  in  the  way 
of  HSI  evaluations  and  inputs  to  the  acquisition  process. 

The  present  study  has  defined  a  process  whereby  HSI  evaluations  can  be 
planned,  conducted,  analyzed,  and  presented  to  decision-makers.  Products 
currently  available  for  conducting  analyses  were  reviewed  and  included  in  a 
searchable  database,  scheduled  for  release  via  the  internet  on  the  NPS  HSI 
homepage.  Software  for  defining  the  interactions  between  relevant  parameters 
of  HSI,  as  well  as  an  interface  to  display  potential  effects  of  program  changes  to 
decision-makers  was  designed  and  is  now  in  development.  This  software  will 
also  be  available  via  the  NPS  website  when  completed.  Additionally,  a  plan  for 
distribution  of  such  tools  to  the  acquisition  community  was  discussed.  While 
improvements  to  the  design  of  specific  software  will  always  be  made,  the  basic 
process  for  conducting  HSI  evaluations,  as  well  as  the  individuals  who  should  be 
apprised  of  the  process  and  resources  available,  is  relatively  constant. 
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In  an  era  of  manpower  drawdowns  and  continual  reductions  in  program 
budgets,  recognizing  the  potential  for  system  improvement  through  better 
integration  of  the  human  element  represents  a  veritable  goldmine  of  savings. 
Omission  of  the  burgeoning  concept  of  HSI  will  result  in  wasted  time,  taxpayer 
dollars,  and  potentially,  the  very  lives  of  service  members  themselves. 
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